System and method for generating a virtual reality interface for displaying patient health data

ABSTRACT

The disclosed system generates a three dimensional virtual space that includes an object representation of at least a portion of a human body and a first three-dimensional cylindrical surface floating within the three-dimensional space, wherein the first three-dimensional cylindrical surface includes a two-dimensional data area for a presentation of data to a user viewing the first three-dimensional cylindrical surface in the three-dimensional space. A two-dimensional data representation of first physiological data is displayed on the two-dimensional data area. In response to receiving a user selection of a portion of the three-dimensional cylindrical surface, the system generates one or more additional surfaces floating within the three-dimensional space. A two-dimensional data representation of second physiological data associated with the first physiological data is displayed on a data area of the one or more additional surfaces.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/828,365, entitled “GRAPHICAL PATIENT AND PATIENTPOPULATION DATA DISPLAY ENVIRONMENT AND ELEMENTS,” filed Apr. 2, 2019,which is hereby incorporated herein by reference in its entirety andmade part of the present U.S. Utility patent application for allpurposes

BACKGROUND FIELD

The subject technology addresses deficiencies commonly encountered inhospital care and medical care user and system interfaces and display ofpatient and patient population-based health data.

SUMMARY

Existing systems and visual presentations (e.g., screen layouts) are notconducive to consolidation and simplification of critical patientinformation and/or comprehensive visual presentation of a patient orpatient population condition with time-scrubbing capability andpresentation of event-related data. For example, for a clinician toascertain a patient condition from data residing in multiple devices orsystems, the clinician may interact with or navigate each of thesesystems or data sources separately and then integrate the data sourcesmanually or read these data from dashboard-style visual elements, and/orthrough navigation of multiple screens, systems, windows, panes, listsor combinations thereof. Furthermore, the ability to time-relate eventsor data elements is not straightforward nor enabled by thesevisualization tools. For example, if a clinician notes that a ventilatorparameter was in a state that required attention at a certain time orwas in a range which was flagged/marked for clinical reasons, if theclinician desires to see what was happening in the patient's lab data(e.g., blood panel) at that same time point the clinician may have tosearch for that data manually and once found, that data may not beprovided in a manner that would facilitate viewing alongside or in sightof the associated ventilator event. Time sequences of medical patientdata or patient population data are also not typically presented in agraphical way that can be manipulated appropriately within the contextof other medical data sources that may or may not be related.

Challenges associated with presenting multiple data elements frommultiple data sources in a single collection of visual elements aremagnified by the two-dimensionality of most visual presentations. Forexample, presentation of time trends of patient data are typicallyexecuted in a linear or rectangular fashion which frequently leads tocrowding of visual elements and unreadable small data plots asadditional data sources are added to a fixed screen or visualpresentation space (e.g., monitor, tablet screen). Accordingly, there isa need for an improved visual presentation layer or element andmethodology of presentation for patient health and population healthdata which can remedy these deficiencies.

In one aspect, the subject technology provides three dimensional visualelements and structures for presentation of patient health andpopulation health data. In this regard, the subject technology comprisesa three dimensional visual health data element that combinestime-related health data for individual patient diagnostic and/ortherapeutic systems (e.g., ventilation, laboratory tests, medicationinfusion, etc.). The health data element is visualized in the form of arotatable, extendable, and configurable cylindrical structure. Thisstructure provides for visual time-alignment of critical patient dataand patient flags/markers related to important aspects of a patientcondition. For example, markers related to weaning from ventilation,sedation levels, lung protection, and others are visually combined bythe subject technology to transform separate data sources into a singlecomprehensive visual indicator of the patient condition or patientpopulation status. According to various aspects, the visual cylindricalhealth data element substantially surrounds an object representation ofat least a portion of a human body—which together provide an integratedvisual touch point and flag/marker/event indicator. The objectrepresentation may include, for example, a visual representation of therelated patient. Markers/flags/events are defined within each system forthe patient (e.g., respiratory, sedation, laboratory, infusion, vitals)and represent patient data points which are either outside of normalvalues, outside of defined protocols, not within certain thresholds orconversely within certain important clinical bounds (e.g., for events).As a clinician, it is essential to get a holistic view of a patient'scondition and the cylindrical structure or structures of this disclosureenable this centralized view.

According to various implementations, the disclosed system generates athree dimensional virtual space that includes an object representationof at least a portion of a human body and a first three-dimensionalcylindrical surface floating within the three-dimensional space, whereinthe first three-dimensional cylindrical surface includes atwo-dimensional data area for a presentation of data to a user viewingthe first three-dimensional cylindrical surface in the three-dimensionalspace. A two-dimensional data representation of first physiological datais displayed on the two-dimensional data area. In response to receivinga user selection of a portion of the three-dimensional cylindricalsurface, the system generates one or more additional surfaces floatingwithin the three-dimensional space. A two-dimensional datarepresentation of second physiological data associated with the firstphysiological data is displayed on a data area of the one or moreadditional surfaces.

In some implementations, the one or more additional surfaces include oneor more additional three-dimensional surfaces and, in response toreceiving the user selection, the one or more additionalthree-dimensional cylindrical surfaces floating within thethree-dimensional space are generated. A two-dimensional datarepresentation of second physiological data associated with the firstphysiological data may be displayed on a data area of the one or moreadditional three-dimensional cylindrical surfaces. The one or moreadditional three-dimensional cylindrical surfaces may be displayed aboveor below and adjacent to the first three-dimensional cylindricalsurface, with each additional three-dimensional cylindrical surfacebeing circumscribed about the portion of the object representation.Other aspects include corresponding systems, apparatuses, and computerprogram products for implementation of the computer-implemented method.

Further aspects of the subject technology, features, and advantages, aswell as the structure and operation of various aspects of the subjecttechnology are described in detail below with reference to accompanyingdrawings.

DESCRIPTION OF THE FIGURES

Various objects, features, and advantages of the present disclosure canbe more fully appreciated with reference to the following detaileddescription when considered in connection with the following drawings,in which like reference numerals identify like elements. The followingdrawings are for the purpose of illustration only and are not intendedto be limiting of this disclosure, the scope of which is set forth inthe claims that follow.

FIG. 1 depicts example virtual reality interface for the display ofpatient health data, including visual cylindrical health data elementssubstantially surrounding a visual representation of a patient, witheach element represented as a cylindrical band with health-relatedflags/markers superimposed thereon, according to various aspects of thesubject technology.

FIG. 2 depicts a further virtual reality interface, includingcylindrical bands with data flags/markers, according to various aspectsof the subject technology.

FIG. 3 depicts a further virtual reality interface, includingcylindrical bands with data and flags/markers and neonatal visualpatient, according to various aspects of the subject technology.

FIG. 4 depicts an example virtual reality interface, including asegmented cylindrical health data element surrounding a patient at asingle raised level, with a selected segmented band providing furtherdata panels/bands to display additional corresponding patient data,according to various aspects of the subject technology.

FIG. 5 depicts an example virtual reality interface, including asegmented cylindrical health data element surrounding a patient at asingle base level, with a selected segmented band providing further datapanels/bands to display additional corresponding patient data, accordingto various aspects of the subject technology.

FIG. 6 depicts an example graphical interface, including a cylindricaldata element surrounding a two dimensional image, the data elementfunctioning as an image selector and displayer with rotational time,according to various aspects of the subject technology.

FIG. 7 depicts an example graphical interface, including cylindricalstructures for displaying labs, vitals, and images, according to variousaspects of the subject technology.

FIG. 8 depicts an example graphical interface, including a cylindricalselection wheel for display metrics/data streams, according to variousaspects of the subject technology.

FIGS. 9A and 9B depict examples of a virtual reality interface,including a patient population view of multiple patient representations,each with cylindrical bands displaying example markers, according tovarious aspects of the subject technology.

FIG. 10 depicts an example virtual reality interface, including anindividual patient view with cylindrical bands displaying respirationand sedation markers, alarms compliance, time-series data, and protocoladherence, according to various aspects of the subject technology.

FIG. 11 depicts a further example virtual reality interface, includingcylindrical bands with data flags/markers, according to various aspectsof the subject technology.

FIGS. 12A, 12B, and 12C depict further examples of virtual realityinterface, including cylindrical bands with data flags/markers,according to various aspects of the subject technology.

FIGS. 13A and 13B depict examples of a virtual reality interface,including a patient representation with one or more cylindrical bandsdisplaying example data and markers, according to various aspects of thesubject technology.

FIGS. 14A and 14B depict example alternative views of a cylindricalband, according to various aspects of the subject technology.

FIG. 15 depicts an example process for generating a virtual realityinterface for the display of patient health data, according to variousaspects of the subject technology.

FIG. 16 is a conceptual diagram illustrating an example server-clientsystem for providing the disclosed virtual reality interface for thedisplay of patient health data, according to various aspects of thesubject technology.

FIG. 17 is a conceptual diagram illustrating an example electronicsystem for generating a virtual reality interface for the display ofpatient health data, according to various aspects of the subjecttechnology.

DESCRIPTION

While aspects of the subject technology are described herein withreference to illustrative examples for particular applications, itshould be understood that the subject technology is not limited to thoseparticular applications. Those skilled in the art with access to theteachings provided herein will recognize additional modifications,applications, and aspects within the scope thereof and additional fieldsin which the subject technology would be of significant utility.

The subject technology addresses various challenges related to managingpatients that exist in hospital and other medical environments where alot of different data sources are available or required to make clinicaldecisions. Multiple devices may contribute to a patient's care. Forexample, the intensive care unit may utilize ventilators, infusionpumps, cardiac equipment, and various sensors. Additionally, cliniciansmay rely on a substantial amount of lab data in the form of, forexample, patient electronic medical records, test results, etc.Clinicians (e.g., respiratory therapists, nurses, doctors) are oftenforced to search for information in different systems to put together astory about a patient to navigate and manage patient care. Each type ofdata and/or system may be viewed in isolation in one particular areawithout a way to time align or assimilate with other data feeds.

The subject technology addresses the foregoing problems by aggregatingdisparate data into a three dimensional (3D) centralized representationof patient data structures, and which allows a clinician to visualizepatient data in a time-aligned manner and in a way that indicates howeach portion of data (retrieved from the multiple data sources) relateto each other. Thus, a clinician may view all of the data without havingto switch between different data sources, or between panels or screensor tabs, as is done in traditional systems. The subject technologyprovides the data superimposed onto a 3D cylindrical structure, within avirtual 3D space. The 3D cylindrical structure may include multiplecylinders, with each cylinder providing one or more data sets for reviewin the 3D space. Each cylinder can be rotated infinitely to review, inembodiments, a time-aligned data set provided by the cylinder, andmultiple cylinders may be stacked as bands or rings of different datasets on top of one another.

FIGS. 1 through 5 depict various views of an example virtual realityinterface for the display of patient health data, including visualcylindrical health data elements substantially surrounding a visualrepresentation of a patient, with each element represented as a threedimensional floating cylindrical band with health-related flags/markerssuperimposed thereon. For the purpose of this disclosure, the termscylindrical “elements” and cylindrical “bands” may be usedinterchangeably when describing the disclosed cylindrical structuresfloating in the 3D space.

The virtual reality interface described herein is implemented bycomputer software and visually presented as three dimensional objectswithin a three dimensional space. According to various aspects, thedisclosed three dimensional cylindrical data elements may be manipulatedin the three dimensional space by user input. The user input may be, forexample, by way of a touch screen to manipulate the elements by sensinguser input on an area of the touch screen corresponding to the displayof the elements. The virtual reality interface may also be visuallypresented in an augmented reality system, whereby one or more users mayview the virtual reality interface using augmented reality viewingtechnology. With brief reference to FIGS. 16 and 17 , the virtualreality interfaces depicted herein may be displayed on a display screen1706 of a computing device 1700 operably connected to the centralizedserver 1710. Patient data is stored in one or more data sources (e.g., adatabase) in real time and managed by a centralized server 1710. Thecentralized server aggregates the real time patient data and providesthe data for display by computing device 1700 in real time.

FIG. 1 depicts an example virtual reality interface 100 for the displayof patient health data. A visual cylindrical data element 110, visualcylindrical data element 115, and visual cylindrical data element 120are included in the depicted virtual reality interface 100.

Each cylindrical data element 110, 115, 120 is displayed responsive to aprior selection of data and corresponds to the data selected. Forexample, data element 110 includes patient vitals data, data element 115includes lab data, and data element 120 includes respiratory data. Eachdata element may be configured to obtain its data from a respective datasource or multiple data sources. According to various implementations,interface 100 may include a configuration screen (not shown) whichallows a user to indicate and/or or select which types of data may bedisplayed on interface 100. Interface 100 may then render respectivecylindrical data elements for the respective selected data types. Insome implementations, the cylindrical data elements may be configured orreconfigured to display a particular type of data based on apredetermined type of user selection (e.g., a right click, ortap-and-hold gesture) of the element within the 3D space which bringsabout a dialog for selection between available data types. In someimplementations, the cylindrical data elements may be configured todisplay data based upon a user profile, clinical specialty, patienttype, disease state, facility type or care area.

Each of the aforementioned individual cylindrical data elements displaytime-related patient data that include flags or markers which may bedisplayed alongside of the patient data for viewing. For example, aflag/marker 125 is included in the visual cylindrical data element 110,and a flag/marker 130 is included in the visual cylindrical data element115. For the purpose of this disclosure, the terms “flag”/“flagged” and“marker”/“marked” may be used interchangeably. According to variousimplementations, flagged (or marked) portions on the 3D cylinders mayindicate data points for further review. Data can be marked by way of agesture, or marked automatically when the data satisfies or exceeds athreshold, or predetermined range. A marker may appear on a cylinder,behind corresponding data, in a different color (e.g., green, yellow,red, etc.) than the data and/or the cylinder. In this regard, thecylinder, data, and/or marker itself may be colored to indicate a levelof importance or the data source from which the data came from. Asdepicted in FIG. 1 , a marker may appear as a vertical stripe (e.g., inred) and have a thickness corresponding to a time element of the data.

A marker may be selected, for example, by a gesture at or near themarker such as a tap on the marker or a swipe over the marker. As willbe described further, selecting a marker may expand the data at themarker. In this regard, further data representative of the marker may bedisplayed on another cylindrical band that appears above or below thecurrent band, or may be displayed on the current band or proximate tothe current band as a separate callout or dialog of data. Additionallyor in the alternative, selecting a marker specific to one data stream(e.g., volume or pressure, heart rate, etc.) and that corresponds toindividual data that's flagged or out of range may cause the display ofa secondary band on the band that is colored underneath the data set.

Each cylindrical data element 110, 115, 120 may take the form of asemi-circle or, according to some aspects, may take the form of a closedor complete circle or ellipse. The axis of cylindrical data element 110,115, 120 may define a point of rotation such that the element may becaused to freely rotate about the axis (from right to left or left toright). As depicted in FIGS. 1 through 5 , virtual interface 100 mayinclude a graphical slider 132 which, when activated in a direction(e.g., slid horizontally to the left or right), will rotate thedisplayed cylinders according to the direction the graphical slider 132is moved. In some implementations, only cylinders that are preselectedmay rotate as slider 132 is moved. Selection may be by way of touchgesture to highlight a cylinder prior to selection or by activating thecylinder for rotation by way of setting a visual flag corresponding tothe cylinder (e.g., a checkbox). In this regard, a cylinder to berotated by slider 132 may be locked or unlocked to rotate with orindependently of, respectively, another cylinder, as described furtherbelow. Each cylindrical data element may circumscribe an objectrepresentative of at least a portion of a human body. As depicted inFIGS. 1 through 5 , the object may be a 3D representation of a humanbody 140.

Virtual reality interface 100 may further include a virtual listing 150of one or more selected patient data values. Virtual listing 150 mayappear within the 3D space over the rings and/or to a side of the object140. In some implementations, the virtual listing 150 may replace object140 on the axis of the cylindrical data elements. Each patient datavalue displayed in virtual listing 150 may be a current value of ameasured physiological parameter (e.g., last measured blood pressure,temperature, oxygen level, etc.). In this regard, a patient data valuemay change color or become highlighted or otherwise flagged upon thedata value satisfying a predetermined threshold (e.g., exceeding a limitfor the physiological parameter). The patient data values displayed invirtual listing 150 may be selected by a user or, in someimplementations, automatically display based on the display of acylindrical element displaying the corresponding data for the datavalue. For example, cylindrical data element 120 may display a real timeinspiratory flow over a period of time, and the current correspondingdata value of the real time inspiratory flow may be displayed in listing150.

In some implementations, the representation of the human body 140 (orportion thereof) may appear adjacent to the cylindrical structure ratherthan surrounded by the cylindrical structure, and the body may also moveinto or out of the virtual space based upon data visualization. In someimplementations, object representation 140 may be manually orautomatically (e.g., based on machine learning) zoomed to a portion thatis relevant to the data being shown, or may be replaced by only aportion of the body relevant to the data (or marker, as described below)for more detailed visualization. For example, if respiratory data isbeing reviewed, a full body may be replaced by just the chest region,including the lungs, to show in greater detail the areas of concern orinterest, or that correspond to a given therapy or generated markers.

FIG. 2 depicts a further example virtual reality interface 200,including cylindrical bands with data flags/markers, according tovarious aspects of the subject technology. The example of FIG. 2includes additional graphical elements to those discussed in FIG. 1above. As illustrated, FIG. 2 includes additional three-dimensionalcylindrical surfaces 210, 215, 225, and 230 floating within athree-dimensional space of the virtual reality interface 200. FIG. 2 isdiscussed in more detail below.

FIG. 3 depicts a further example virtual reality interface 300,including cylindrical bands with data and flags/markers and neonatalpatient representation, according to various aspects of the subjecttechnology. As illustrated, a visual cylindrical data element 340, andvisual cylindrical data element 345 are included in the virtual realityinterface 300. FIG. 3 includes additional three-dimensional cylindricalsurfaces 310, 315, 325, 330, and 335 floating within a three-dimensionalspace of the virtual reality interface 300. FIG. 3 is discussed in moredetail below.

FIG. 4 depicts a further example virtual reality interface 400,including a segmented cylindrical health data element 410 surrounding apatient at a single raised level, with a selected segmented bandproviding further data panels/bands to display additional correspondingpatient data, according to various aspects of the subject technology. Inthe depicted example, the segmented bands circumscribe an axis in the 3Dspace, with each band being separated from each other by an open space.In this regard, each segmented band may display a different type of data(e.g., from a different data source). A segmented band 420 may beselected by a user to generate further data panels/bands 421, 422, 423,424, 425 below or above it for the display of additional correspondingpatient data. The newly displayed bands and data may be associated withthe selected band. For example, when a user selection is received at afirst band of the segmented bands, the system may be configured todisplay one or more additional three-dimensional cylindrical surfaces tobe displayed above or below the first band in a curvilinear planealigned with a curvilinear plane of the first band. In one example, band420 may include a variety of event flags over a period of time and, byusing a predetermined gesture (e.g., a tap-hold-and-drag), a user maycause the display of bands 421, 422, 423, 424, 425 which eachrespectively display data that corresponds to each respective data flagindicated in band 420.

FIG. 5 depicts a further example virtual reality interface 500,including a segmented cylindrical health data element 510 surrounding apatient at a single base level, with a selected segmented band providingfurther data panels/bands 520, 522, and 524 to display additionalcorresponding patient data, according to various aspects of the subjecttechnology. As further illustrated, the data panel 520 includes asecondary band 530 that is colored underneath the data set shown in thedata panel 520. Since each cylindrical data element may be time based inthat the data displayed by the element is displayed across the elementaccording to a time scale, secondary band 530 may represent a period oftime according to the length of secondary band across the same timescale. Secondary band 530 may indicate a section of data that has beenflagged for further review (e.g., by the user or by the disclosedmachine learning algorithm) or may indicate that an event took placeduring the flagged period of time. Also, the data panel 522 includes asecondary band 530 that is colored underneath the data set shown in thedata panel 522.

As described previously with reference to FIGS. 1 through 5 , a visualcylindrical data element is constructed of cylindrical segments or bandswhich can display health-system, body-system or device data, flags,images, events and can be static in time or can be time-scaled. Thecylindrical bands can be manipulated individually or in groups, can betime-locked to other cylinders, can be re-arranged, swapped orcustomized. According to various implementations, when a band becomestime-locked, the band may become vertically aligned in time such that iftwo bands are viewed together they are using the same time scale andthey are aligned by time such that if one band is rotated to scrollthrough time, the other bands will rotate along with it to stay inalignment so events may be viewed in different systems (e.g., bands)that occurred at the same time. In this regard, if a band is unlocked,it can be rotated independently of the other bands and will be out oftime-alignment until re-locked in which case it will rotate back intotime alignment with other bands.

Individual cylindrical bands which display time-related patient data canadditionally contain flags or markers which are displayed alongside thedata for easy viewing. Implementations of the visual structure can alsocontain a separated or integrated flag/marker/event advance button oricon which allows the user to quickly move through patientflags/markers/events by advancing incrementally through flags with otherdata or system bands moving in time alignment. Alternatively, the bandscan move independently using the advance button if the cylindrical bandstack is unlocked. In this case only a highlighted band may advance fromflag/marker/event to flag/marker/event while other bands may remainstationary with current values. According to various implementations,advancement between markers/events/flags can also be achieved using aselector wheel or other rotational structure or band.

Cylindrical segments/bands can represent any time period of interest,for example, the time from admission to the current time or the previous24 hours. The time period may be selectable using a selection cylinderor other selection means to determine a time-period of interest forvisualization. Visual indicators for markers/flags/events can becomprised of lines or solid bars or tabs with magnitudes that correspondto the time a value is marked. Alternative visual markers are alsowithin the scope of this disclosure. If a user is interested in seeingtrends of particular values, the user can expand one or more bands of agiven system into individual cylindrical segments which contain trendgraphs for the time period of interest. As an example, a user interestedin viewing ventilation or respiratory-related patient data can select astack of cylindrical bands with PEEP, respiratory rate, tidal volume,EtCO2 etc. Alternatively, trend graphs for data from different systemscan be mixed and matched. The cylinder band stack can be customizable orcan be selected from defaults or alternatively can be automaticallystacked and selected by intelligent rules that present the user with themost appropriate patient information relevant to their currentcondition. The height and circumferential extent of the visualcylindrical element can be infinite as values can be populated onto thebands as the bands come into view during rotation or when the cylinderis moved up or down. Individual cylindrical bands or elements can alsobe manipulated in size to amplify views, zoom into data sets or images.As a user increases the height of a system cylindrical segment, as theavailable space to display data increases, so too does the fidelity ofthe data (e.g., cylindrical segment band representing FiO2 withflags/markers can become a FiO2 vs. time graph as the cylindricalsegment height increases).

A cylindrical band may display a plot or a graph of some data versustime. A clinician may want to increase the size of an interestingportion of the data to determine a more accurate picture patient data orvitals. Virtually touching or dragging the data or the cylinder over thedata may expand the height and/or width of the cylinder and/or thecorresponding data. In some implementations, as described further withregard to FIG. 14A, for data types which may be more easily viewed in aflat orientation (e.g. images, graphs, tables), one or more selectedcylinders may unfold or unwrap into a more planar view. In someimplementations, plots may be overlayed. The system may be configured toallow a clinician to perform a gesture or dragging motion in which oneor more parameters are moved or dragged onto an area in the 3D space,and immediately appear as a new plot or graph. In some examples, newdata may be dragged from multiple bands and co-plotted with multipleaxes, if necessary, and a common time axis on the bottom.

The cylindrical bands may also be reordered by dragging or movinggestures. Two data sets may be combined by dragging one down data setdisplayed on a first band onto another second band and holding it there.Two cylindrical bands may also be combined by grabbing each of them witha finger and then pinching them together. The data may then be displayedtogether on the same band, for example, using a common time signature.Bands that include multiple data sets may be expanded or the data setsseparated using a reverse pinch gesture. In some implementations, asecondary icon may float in the 3D space near the structure. The iconmay include a symbol indicating the icon may be used to combine data.When a clinician drags one or more band towards the icon the system maycombines the data with other data dragged to the icon, or create a newband with the data displayed on it.

Markers may also indicate start or end points. For example, for apatient being weaned off of ventilation, a marker may indicate the startof a sedation vacation or sedation awakening trial where the patient'ssedation levels are reduced, or may indicate a spontaneous breathingtrial where ventilator assistance is reduced to see if the patient maystart to breath on his or her own. Markers may also indicate a point intime corresponding to when a patient missed a breathing trial, mayindicate when a breathing trial was started late, the breathing trialwasn't coordinated with the sedation reduction, etc.

According to some implementations, the system may include machinelearning algorithms to look for patterns in data and project what stepsshould be taken with regard to patient care. Markers may beautomatically generated that correspond to future events or actions thatshould be performed. For example, if some event X has occurred in thepast, then the machine may determine that Y will occur at a future time.Depending on the event, the machine may mark the future time point,signal a review of data at that specific future time point, and signal alikelihood of an event happening or signal an alert for a patient if thelikelihood is a significant or morbid/undesirable event. In anotherexample, machine learning may identify likely patient paths from a givenpattern of data and, in some implementations, automatically make its ownadjustments to care systems (e.g., as an autopilot). For example, onidentifying a flow rate of ventilation is not providing a desired effecton the patient, the flow rate may be adjusted (e.g., withinpredetermined limits) to move the patient towards the desired effect. Insome implementations, a caregiver may be provided an alert by thesystem. An example alert may be sent to a caregiver device, and includea text message indicating “heads up, your patient X is trending towardsa ventilator-associated event” or “heads up, your patient Y is ready forweaning from ventilation.”

If a user is observing the visual cylindrical health data element andsees a flag/marker/event on the ventilation/respiratory segment, theuser can select that flag/marker/event. In order to provide acomprehensive picture of the patient's health including possiblecontributing factors in other systems, the cylindrical band/segment ofchoice (e.g., ventilation) may be rotated to bring the selected flag tothe front of the cylindrical element closest to the user. At the sametime, adjacent cylindrical segments related to other systems mayautomatically rotate and time align with that flag to provide a visualrepresentation of what was happening with the patient in other systemsat the same time point. As described further below with regard to FIG.14B, as a band rotates and/or is being selected or interrogated for dataor a marker, the front of the band may contain a band magnifier thatenlarges a portion (e.g., the front piece) of the band for betterviewing. In this regard, the band magnifier may both enlarge and timecompress data.

Selection of screen elements is not limited to physical touch or keystrokes on a keyboard or similar device but rather should be taken toencompass all possible selection methods including methods encompassedin virtual and augmented reality environments such as gesturing througheye movements or other body movements. The rotation of the visualcylindrical health data element enables a 360-degree view of the patienthealth condition and provides an immersive environment which throughaugmented reality and/or virtual reality may enable a user to exploremore data in a centralized environment. For example, a tablet can beused with augmented reality to see significant amounts of patient databy enabling a user to walk around the visual health data cylinder andview it from all 360 degrees. Similarly, if wearing augmented realityglasses or virtual reality glasses (or similar imaging tools), one ormore users can walk around the cylindrical health data element, gatheraround the element with other users or clinicians, and even spin andmanipulate the cylinder in an interactive way to provide views to otherusers.

As depicted in FIGS. 1 through 5 , the system may display an objectrepresentation of at least a portion of a human body. This objectrepresentation may be individualized for the patient based on thepatient's profile and/or medical history. According to various aspects,the object representation may include a 3D human body and/or may includerepresentations of various organs within the human body (e.g., lungs,heart, intestines, reproductive system according to the gender of thepatient, etc.). For example, a human body representation may bedisplayed as male or female, or child or adult, be anatomically correct,and be generated based on demographic information, hair color, skintone, weight, height, etc. According to various implementations, therepresentation may be displayed in the center of the cylindrical bands(e.g., along the axis of the bands) or may appear adjacent to thecylindrical bands. Portions of the 3D human body or the organs withinthe body may be illuminated to indicate relevance to a portion of data.In some implementations, portions of the body and/or organs within thebody may be highlighted—and corresponding cylindrical bands of datagenerated—based on preexisting conditions determined from the patient'smedical history or from events or conditions currently occurring withthe body.

The system may display more than one human, for example, transplantpatients, pregnant patients, and multiple data sets or overlapping datasets could be shared and viewed together when there is a commonalitybetween more than one patient.

For example, flashing or illuminated lungs may indicate that the dataincludes one or more respiratory related markers. A relevant portion ofthe human body (e.g., lungs) may also illuminate or flash when dataand/or a cylindrical band is selected that is representative of thatportion of the human body (e.g., selecting ventilation data). In someimplementations, the body may not stay as a fixed whole body. Forexample, the hollow outer body may disappear and the relevant portion ofthe human body (e.g., the lungs) may animate and grow in size to floatin the 3D space in place of the outer body in response to certain eventsor actions, or selection of the related data. If the system detects aleft lung injury or a lesion or a problem with something that's beenplaced inside the lung (e.g., from a lab or radiology report) the leftlung may enlarge into the center of the view to highlight the detectedissue. Selection of the radiology report or relevant data in the report,the lab, or on a cylindrical band may cause the corresponding portion ofthe human body to be expanded and/or highlighted.

Providing data on cylindrical bands surrounding portions of the humanbody relevant to that data provides the data in a more familiarenvironment. This way, the clinician doesn't have to think about how thedata correlates to a particular issue. Additionally, the more data thatis input and presented and manipulated, the more that machine learningcan diagnose issues and predict future events. The system mayautomatically determine from a radiology report that there is an issuewith the left lung and illuminate that portion of the lung. Theclinician immediately sees the illuminated lung and knows to look formarkers indicative of the issue. In some implementations, the degree ofillumination (e.g., brightness, whether flashing, etc.) may bedetermined by the severity of the issue detected. The system (e.g., arules engine) constantly interrogates markers and prioritizes issuesthat need to be looked at by a clinician. According to various aspects,this visual layer may operate with data that is coming in through anintegration engine which is collecting data from individual devices, orthrough a hub connected to the individual devices. A knowledge layer,such as a rules engine and/or a machine-learning algorithm, mayinterrogate and/or analyze the data for markers, events, patterns,conditions, and the like. The system may automatically identify hotspots and bring up additional views or flag portions of the body in 3Din response to the prioritization. In some implementations, the systemmay also display textual messages to the clinician to alert them tomarkers. In some implementations, the system may generate an instructionor suggested action or provide a description of a marker or event forthe clinician. Such textual messages may appear on or adjacent to thecylindrical band which is relevant to the marker or event.

Data drilling or interrogation can occur due to the depth of the core ofthe cylinder and similar to the functionality of the cylinder in beinginfinitely rotatable or vertically extended, the core of the cylinder onits side can be infinite or semi-infinite in that what is available isnot limited to what is seen or what can be presented visually. Theobject representation can also serve as a source of indication relatedto health data or health condition. For example, if a flagged healthdata value exists for a patient, the centralized object representationcan be flashing or can have an alternative method of alerting theclinician or user that there is an issue which should be attended to oracknowledged. When the user selects the displayed body, the visualcylindrical health data element can automatically display the systemsegment which contains the flag aligned to the front position and mayadditionally time-align other systems to provide a streamlined visualdata path. The indication provided by the object representation may bespecific to a body system (e.g., lungs flashing) or may be generalizedto the entire body. The object representation element may be stationaryor rotating or may have movement of another type which is visuallyhelpful to expose the most comprehensive view of the patient condition.

In some implementations, the representation of the human body may alsobe used to indicate program operations. For example, when the system isactively collecting or acquiring data, the 3D body may rotate and, insome implementations, if the body is not rotating then no active datacollection is going on.

Movement and visual presentation of cylindrical bands and elements canalso be important to user interaction with the patient data—for example,cylindrical bands and elements can rise, fall, rotate, highlight, flash,or change orientation. For example, a cylindrical element can changefrom being viewed in perspective or vertically to being looked at on itsend. In this way, patient health data can be presented or represented indifferent ways. For example, the core of the cylindrical space can beused to display a health data element or medical image while thecircular bounds or outside of the circular bounds can be used torepresent the time sequence of that type of health data as shown in FIG.6 using X-rays as an example.

FIG. 6 depicts an example graphical interface 600, including acylindrical data element 610 surrounding a two-dimensional image area615, with the data element 610 functioning as an image selector anddisplayer with rotational time, according to various aspects of thesubject technology. Some fixed data (e.g., radiology images or a labreport) may be represented on a cylindrical element and not include atime parameter. For example, data displayed on a cylindrical element maybe marked with a radiology or imaging marker. In such cases, selectingthe marker corresponding to the data may cause the display of adifferent type of a visual that's larger and conducive to reading. A newdialog 610, 615 may appear in the 3D space as a floating 2D plane. The2D plane may be positioned with respect to the same axis as thecylindrical band(s) such as to be able to be rotated for display toother clinicians in an augmented reality space. In the depicted example,the dialog is an image viewing dialog that has access to multiple images(e.g., x-rays) and includes a rotatable data element 610, which isrotatable about an image area 615, for selection of one of the multipleimages to display in image area 615. In the depicted example, multiplex-ray images are displayed about the image area 615 for selection, and aselected x-ray image is displayed in image area 615. The graphicalinterface in FIG. 6 may be included in a given virtual reality interfaceas described above in FIGS. 1-5 .

The core of the interaction with the patient population and individualpatient according to this disclosure may be achieved entirely orsubstantially through cylindrical elements. For example, cylindricalelements or cylindrical element orientations other than the cylindricalband structure examples shown in FIGS. 1 through 5 can be utilized forcontrol functions and display functions. For example, when viewinglaboratory slips or vital signs or tabular data or images, it may beundesirable to view such items on an elongated horizontal form such asthat shown in FIGS. 1 through 5 . In this case, virtual cylinders withsmaller aspect ratios can appear or pop out of the centralized datacylinder. These smaller virtual cylinders may be more conducive toviewing these items. FIG. 7 depicts an example graphical interface 700including a first cylindrical structure 710 that functions as a firstimage viewing dialog to display images such as an x-ray of the patient,and a second cylindrical structure 715 that functions as a secondviewing dialog to display patient labs and/or vitals (e.g., in imageand/or textual form). These cylindrical structures 710, 715 are separatefrom the centralized virtual cylinder structure depicted in FIGS. 1through 5 , and may be depicted alone or together with the centralizedvirtual cylinder structure.

With further reference to FIGS. 6 and 7 , the foregoing image viewingdialogs may be configured to display images associated with a patient.In some implementations, an image viewing dialog may include an imageviewing area displaying the first image, and circumscribed by multipleicons representative of selectors for other images to display in theimage viewing area. In response to receiving an icon selection of one ofthese icons, the dialog may be configured (e.g., by instructions) todisplay a second image (corresponding to the selection) in the viewingarea.

FIG. 8 depicts an example graphical interface 800, including acylindrical selection wheel 810 for display metrics/data streams,according to various aspects of the subject technology. According tovarious implementations, a selection icon 805 may be provided on virtualreality interface 100 for the selection of various items, includingbetween available data sets. When selected, the selection icon 805 mayexpand into the depicted cylindrical selection wheel 810. Selectionwheel 810 is rotatable in a vertical direction to select between thevarious data sets. In one example, a predetermined gesture with respectto a cylindrical element may cause virtual reality interface 100 tobriefly display selection icon 805, which may then be activated to useselection wheel 810 to change the data displayed in the cylindricalelement. In another example, the selection icon 805 and correspondingcylindrical selection wheel 810 may be used to select values to displayin virtual listing 150. While cylindrical selection wheel 801 isdepicted as moving in a horizontal direction for selections, cylindricalselection wheel 810 may move in a horizontal direction to perform thesame selections.

FIG. 9A depicts a first example of a virtual reality interface 900,including a patient population view of multiple patient representations,each with cylindrical bands displaying example markers, according tovarious aspects of the subject technology. As illustrated in FIG. 9A,the patient population view includes a graphical representation of acare area patient population, including patient 901, patient 902,patient 903, patient 904, patient 905, and patient 906. As furtherillustrated, cylindrical bands 910 and 912 are provided for the patient901. Cylindrical bands 920, 922, 924, and 926 are provided for thepatient 902. Cylindrical band 930 is provided for the patient 903.Cylindrical bands 940, 942, 944, 946, and 948 are provided for thepatient 904. Further, cylindrical bands 950 and 952 are provided for thepatient 905.

The cylindrical band elements around each patient shown in thepopulation view of FIG. 9A are visual representations of patientmarkers/flags/events relevant to the patient's respiratory and sedationmedication status. In particular, a cylindrical band element around eachpatient is segmented according to markers in each of six categorieswhich include FiO2 (fraction of inspired oxygen), sedation, weaning,lung protection, alarms compliance, and ventilator-associated events(VAEs). In each segment where at least one marker exists, the number ofmarkers is displayed. Above each patient, the total number of markers isalso displayed such that with a quick view, a clinician can determineboth the overall status of their patient population as well as thestatus of any given patient. Segmentation of the cylindrical band iscontrolled by the number of categories with markers such that if apatient has no markers, no cylindrical band may be displayed. Touchingor gesturing or selecting a patient by other possible means (e.g., AR orVR selection) in the Population View takes the clinician/user to anIndividual View (FIG. 10 ).

FIG. 9B depicts a second example of a virtual reality interface 900,including a patient population view of multiple patients representations970, each with cylindrical bands displaying example markers, accordingto various aspects of the subject technology. In this example, eachpatient representation 970 may be at least partially surrounded by oneor more bands 980, with each band indicating a category of data. In someimplementations, bands may display in interface 900 responsive to amarker being generated, for example, by predetermined threshold beingsatisfied. In this regard, a marker may also represent an alarmcondition. A graphical marker label 982 may be visually associated witheach band (e.g., by way of a color coordinated lead or arrow), and maydisplay the number of markers and/or alarms corresponding to the datacategory of each band, and a time that each marker or alarm conditionwas triggered. A visual marker designation 984 may also be displayedwithin interface 900 for each patient, each indicating the total numberof markers that are active and/or visually indicated in the interfacefor the corresponding patient. Where there are more patients than can bedisplayed on a single screen, interface 900 may include a search input986 by which a patient name, identifier, or partial name or identifier,may be entered to initiate a search and display of a patient populationview of patients matching the search results.

The example care area patient population interfaces of FIGS. 9A and 9Bmay also include aspects of the depicted cylindrical elements but viewedfrom above rather than in a perspective or front view. For example, inone implementation, a collection of patients is viewed from above, forexample, looking down upon the top of their heads. Around each patientrepresentation is a circular or cylindrical element which providesindications of flagged or marked values or conditions. The color of thecylindrical element may indicate the status of a system or monitor orflag. Alternatively, the color of the patient representation may providethis indication. Furthermore, information relevant to markers or flagsor important health information may be presented around the patientwithin the cylindrical or circular element space. If a patient withinthis population is selected by means of touch or other means by theuser, a representation of the patient and/or associated markers or flagsmay rise out of the population into the individual cylindrical healthdata element described above (e.g., in FIGS. 1 through 5 ). In this way,the cylindrical health data element is maintained within different viewsstructures which provide either the same or different information,however, the structure of the patient population view provides the userwith a global image of a patient population of interest.

FIG. 10 depicts an example virtual reality interface 1000, including anindividual patient view with cylindrical bands displaying respirationand sedation markers, alarms compliance, time-series data, and protocoladherence, according to various aspects of the subject technology. Asillustrated, a visual cylindrical data element 1020, visual cylindricaldata element 1021, visual cylindrical data element 1022, visualcylindrical data element 1023, visual cylindrical data element 1024, andvisual cylindrical data element 1025 are included in the virtual realityinterface 1000. Each of the visual cylindrical data elementsrespectively provides a visual representation a data stream ofphysiological data associated with a patient 1001.

FIG. 10 contains an implementation of an Individual View in the subjecttechnology. The Individual View consists of a cylindrical health dataelement which is structured as six cylindrical bands which represent thesix aforementioned marker categories. On each cylindrical band, discretemarkers/flags or events are displayed along a timeline, or a continuousdata value (e.g., FiO2 or tidal volume) is presented as a time series,or both of these data types can be displayed simultaneously. Thecylindrical bands can be manipulated/rotated freely using gesturing orcan be advanced forward or backward using a flag advance button. Atime-alignment lock provides a mechanism to toggle between having allcylinders rotate together in a time-aligned fashion or having individualcylinders rotate independently. A second set of cylindrical bands iscontained below the level where the patient representation is standing(i.e., under the floor). Using a swiping gesture, this second set of sixcylindrical bands can be switched with the marker related bands. A usercan select data sources or elements to present on each of these sixbands using a cylindrical graphing element selection wheel similar tothat shown in FIG. 8 . A user may return to the population view at anytime using a population-view button in the view.

According to some implementations, visual cylindrical elements can alsobe joined or interlocked like gears such that moving a visual cylinderin one plane advances or rotates a connected visual cylinder in anotherplane. Similarly, small cylinders can be used to advance largercylinders within the same plane. In this way a cylindrical control canbe used to advance a larger element. Furthermore, ‘gearing’ ratios canbe used to manipulate the cylinders of the subject technology, eitherdirectly or through associated connected cylinders. For example, whenusing a gesture to rotate a cylinder which contains a very large amountof compact data or large number of flags or markers, said gesture can bescaled such that each real unit of motion in gesturing translate to1/10th or 1/xth of said gesture on the cylindrical band or element.Additionally, to improve viewing of data or data graphs, as the userrotates a given data set or trend to the front of the cylinder, thatregion of the cylindrical band can automatically enlarge for improvedviewing.

Variants of the idea or invention comprise alternative ways to structurethe cylinders or interaction of the cylinder with the user, thestructure of the displayed object representation. It should be notedthat the object representation aspect of the visual element is tailoredto the patient of interest, i.e., if the actual patient is a neonatalpatient, so too may be the object representation.

The cylindrical health data element enables the visual presentation ofdata and markers/flags from separate systems in a consolidated, compactcylindrical structure or set of cylindrical structures withtime-alignment of flags and markers which provides time-based visualdata patient or clinical insights. Population-based data and patientviews are provided in a cohesive three-dimensional visual structurewhich makes it possible to view patient data in a simplified visualrather than in the traditional formats of numeric tables, dashboards,data array and lists. This disclosure enables combining of data from avariety of sources to calculate data that is otherwise not available ina single structure. Adjustability of cylinder height and rotationenables unique visual interaction with both data and other clinicianswho may be viewing the same cylindrical structure from otherperspectives or locations. Visual temporal alignment of markers andflags from different hospital and diagnostic systems in a consolidatedgraphical structure (health data cylinder) provides visual consolidationof events and timelines necessary to produce patient data insights notpossible in traditional or current art visual representations.

FIG. 11 depicts a further example virtual reality interface 1100,including cylindrical bands with data flags/markers, according tovarious aspects of the subject technology. In some implementations, the3D perspective of structures within an interface may be changed (e.g.,by a click and drag operation, touch gesture, or the like). Asillustrated in FIG. 11 , the virtual reality interface 1100 includes agraphical representation of a patient 1101, with cylindrical bands 1110and 1120, rotated such that the angle of viewing is from a perspectivebelow the patient 1101.

FIGS. 12A, 12B, and 12C depict further examples of virtual realityinterface 1200, including cylindrical bands with data flags/markers,according to various aspects of the subject technology. As illustratedin FIG. 12A, the virtual reality interface 1200 includes a graphicalrepresentation of a patient 1201. As further illustrated, cylindricalbands 1210 and 1220 are illustrated for the patient 1201.

FIGS. 13A and 13B depict examples of a virtual reality interface 1300,including a patient representation with one or more cylindrical bandsdisplaying example data and markers, according to various aspects of thesubject technology. In this example, patient representation 1302 may beat least partially surrounded by one or more bands 1304, with each bandindicating a category of data. In some implementations, the bands maydisplay in interface 1300 responsive to a marker 1305 being generatedfor a corresponding category of data, for example, when a predeterminedthreshold being satisfied. In this regard, a marker 1305 may alsorepresent an alarm condition. According to various implementations, amarker 1305 may be visually represented in the cylindrical band as ashaded portion corresponding to a portion of the data for which themarker is associated. For example, a marker 1305 may start at a positionon the band corresponding to when the data satisfied (e.g., surpassed) apredetermined limit, and end at a position corresponding to when thedata returned to a stable value (e.g., within the limit) or at aposition corresponding to a current value of the data, for example, whenthe limit remains satisfied.

According to various implementations, a graphical marker label 1306 maybe visually associated with each band (e.g., by way of a colorcoordinated lead or arrow), and may display the number of markers and/oralarms corresponding to the data category of each band, and a time thateach marker or alarm condition was triggered. Label 1306 may alsodisplay a textual description for the marker or alarm. As depicted,label 1306 may also display a title of the data category (e.g., FiO2,Lung Protection, Alarm Compliance, etc.) and a current value of the data(e.g., percentage value, dosage (in CC), or rate (e.g., L/min), etc.).Similarly each band may also be associated with a graphical limitindicator 1308 indicating one or more predetermined limits on the datadisplayed by the corresponding band (e.g., upper and/or lower limits),and a current and/or averaged or mean value of the data displayed for aperiod of time. A visual marker designation 1310 may also be displayedwithin interface 1300 for the patient 1302, indicating the total numberof markers that are active and/or visually displayed for the patient.According to various implementations, interface 1300 may also include aband selector icon 1312 for selecting data categories to be displayedwithin bands in interface 1300. When activated, band selector 1312 maycause the display of a listing of available data categories, which maybe selected by way of a touch or gesture or similar user selectionmethod. As described previously, virtual interface 1300 may include agraphical slider 1312 which, when activated in a direction (e.g., slidhorizontally to the left or right), will rotate the displayed cylindersaccording to the direction the graphical slider 1312 is moved. In thedepicted example of FIG. 13A, the cylinders selected for rotation maymove within the partial circular path displayed, for example, within thebounds of a quarter circle terminating at first and second edges in the3D space.

FIG. 14A depicts a first example alternative view of a cylindrical band,including a data marker displayed on the band, according to variousaspects of the subject technology. In some implementations, cylindersmay unfold or partially unfold/unwrap to form flatter viewing surfaces.A cylindrical band 1402 may be selected to display the data of the bandin an exploded, flat view of the data. On selection, the data may beredisplayed in a two-dimensional interface 1400, and enlarged to displayvarious aspects of the data values, including a current value of thedata and corresponding limits on the data. For example, thevisualization becomes unwrapped, similar to cutting a can or cylindervertically and then unwrapping it into a flat or flatter view.Accordingly, data types that are more appropriately viewed in a flatrather than curved view may be enhanced, while preserving the cylinderstructure in the 3D display. Markers 1305 may also displayed, asdescribed previously.

FIG. 14B depicts a second alternative view of a cylindrical band with aportion of the band magnified, according to various aspects of thesubject technology. In some implementations, as a band 1402 rotatesand/or is being selected or interrogated for data or a marker, the bandmay be displayed together with a band magnifier 1404 that enlarges aportion (e.g., the front piece) of the band for better viewing. The bandmagnifier 1404 may be activated for display in response to, for example,a predetermined gesture, or by way of activating a predeterminedhardware button on either a pointing device or keyboard. The bandmagnifier 1404 is depicted as a box (e.g., in a shape similar to a beltbuckle), but may alternatively be depicted as a circle or other shape.As illustrated in the depicted example, the band magnifier 1404 mayenlarge and/or time compress data for a period of time. The bandmagnifier may, for example, show a blown up view of a predeterminedperiod of time (e.g., 15 minutes) surrounding an event, even if the restof the band is on a larger time scale (e.g., 24 hours).

FIG. 15 depicts an example process for generating a virtual realityinterface for the display of patient health data, according to aspectsof the subject technology. For explanatory purposes, the various blocksof example process 1500 are described herein with reference to FIGS.1-14 , and the components and/or processes described herein. The one ormore of the blocks of process 1500 may be implemented, for example, by acomputing device, including a processor and other components utilized bythe device. In some implementations, one or more of the blocks may beimplemented apart from other blocks, and by one or more differentprocessors or devices. Further for explanatory purposes, the blocks ofexample process 1500 are described as occurring in serial, or linearly.However, multiple blocks of example process 1500 may occur in parallel.In addition, the blocks of example process 1500 need not be performed inthe order shown and/or one or more of the blocks of example process 1500need not be performed.

In the depicted example flow diagram, a three dimensional space isgenerated, including an object associated with a human body and a firstthree-dimensional cylindrical surface floating within the threedimensional space (1502). According to various implementations, thethree-dimensional cylindrical surface has a height dimension and alength dimension circumscribed about at least a portion of the object,the height and length together defining a two-dimensional data area forthe presentation of data to a user viewing the three-dimensionalcylindrical surface in the three-dimensional space.

Concurrently with generating the first three-dimensional cylindricalsurface, or shortly thereafter, displaying a two-dimensional datarepresentation of first physiological data is displayed on thetwo-dimensional area (1504). The first physiological data may includeone or more events associated with the first physiological data. Forexample, as depicted in FIG. 1 , the first physiological data mayinclude events that correspond to data satisfying or exceeding apredetermined threshold (e.g., an FiO2 level being out of bounds). Auser selection of a portion of the three-dimensional surface is received(1506). The portion of the three-dimensional surface selected maydirectly correspond to an event displayed on the two-dimensional area.In response to receiving the user selection, one or more additionalthree-dimensional cylindrical surfaces is generated, floating within thethree-dimensional space (1508). According to various examples, theadditional three-dimensional cylindrical surfaces are displayed above orbelow and adjacent to the first three-dimensional cylindrical surface,each additional three-dimensional cylindrical surface beingcircumscribed about the portion of the object. Concurrently withgenerating the one or more additional three-dimensional cylindricalsurfaces, or shortly thereafter, a two-dimensional data representationof second physiological data associated with the first physiologicaldata is displayed on a data area of the one or more additionalthree-dimensional cylindrical surfaces (1510).

The first and/or second physiological data may be real time dataassociated with a patient. For example, a data stream corresponding to aphysiological parameter of a patient may be received. Thetwo-dimensional data representation of the first and/or secondphysiological data may be a visualization of at least a portion of thedata stream. In one or more implementations, the first physiologicaldata may be events that correspond to thresholds being met in the secondphysiological data (or vice versa).

Many aspects of the above-described example 1500, and related featuresand applications, may also be implemented as software processes that arespecified as a set of instructions recorded on a computer readablestorage medium (also referred to as computer readable medium), and maybe executed automatically (e.g., without user intervention). When theseinstructions are executed by one or more processing unit(s) (e.g., oneor more processors, cores of processors, or other processing units),they cause the processing unit(s) to perform the actions indicated inthe instructions. Examples of computer readable media include, but arenot limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs,etc. The computer readable media does not include carrier waves andelectronic signals passing wirelessly or over wired connections.

The term “software” is meant to include, where appropriate, firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also, insome implementations, multiple software aspects of the subjectdisclosure can be implemented as sub-parts of a larger program whileremaining distinct software aspects of the subject disclosure. In someimplementations, multiple software aspects can also be implemented asseparate programs. Finally, any combination of separate programs thattogether implement a software aspect described here is within the scopeof the subject disclosure. In some implementations, the softwareprograms, when installed to operate on one or more electronic systems,define one or more specific machine implementations that execute andperform the operations of the software programs.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

FIG. 16 is a conceptual diagram illustrating an example distributedserver-client system 1600 for providing the disclosed virtual realityinterface for the display of patient health data, according to variousaspects of the subject technology. System 1600 includes (among otherequipment) a centralized server 1602 and at least one data storagelocation 1604. Centralized server 1602 and data storage(s) 1604 mayinclude multiple computing devices distributed over a local or wide areanetwork 1606, or may be combined in a single device. Patient data may bereceived, for example, from one or more medical devices 1608 thatcommunicate patient data, over network 1606, to centralized server 1602in real time as the data is collected or measured from the patient atvarious locations in a healthcare organization. Centralized server 1602may store the patient data in real time in data source(s) 1604.

According to various implementations, centralized server 1602 isconfigured to (by way of instructions) generate and provide virtualinterface 100 to clinician devices 1610. In some implementations,centralized server 1602 may function as a web server, and virtualinterface 100 may rendered from a website provided by server 1602.According to various implementations, centralized server 1602 mayaggregate real time patient data (e.g., from data storage 1604) andprovide the data for display in virtual interface 100. The data and/orvirtual interface 100 may be provided (e.g., transmitted) to eachclinician device 1610, and each clinician device 1610 may include asoftware client program or other instructions configured to, whenexecuted by one or more processors of the device, render and displayvirtual interface 100 with the corresponding data. The depictedclinician devices 1610 may include personal computer or a mobile devicesuch as a smartphone, tablet computer, laptop, PDA, an augmented realitydevice, a wearable such as a watch or band or glasses, or combinationthereof, or other touch screen or television with one or more processorsembedded therein or coupled thereto, or any other sort ofcomputer-related electronic device having network connectivity.

According to some implementations, the disclosed 3D cylindricalstructures 1612 of virtual interface 100 may be viewed as part of anaugmented reality, for example, as a holographic set of cylindersstacked over a location within a room 1614. As described previously, arepresentation of the body (or relevant portion thereof) may bedisplayed within the cylinder structure. Clinicians in the room may viewthe cylindrical structure through a set of augmented reality glasses (orgoggles) 1616 (which may be part of or connected to a clinician device1610), or by way of a camera interface of a clinician device 1610. Anaugmented reality device 1616 include various hardware devices, such asa head-mounted display that places images of both the physical world andvirtual objects over the user's field of view, eyeglasses that employcameras to intercept the real world view and re-display an augmentedview through the eyeglass, a heads-up display, contact lenses thatcontain elements for display embedded in to the lens, virtual retinaldisplay in which a display is scanned directly onto the retina or auser, an eye tap, that intercepts light and augments light that passesthrough the center of the lens of the eye of the wearer, and othersimilar devices.

A virtual reality enabled system to augment the disclosed interface formultiple users may include one or more sensors. For example, the system1600 may include one or more positional sensors 1618 that detectindividuals and related motion within the room 1614, and that detectgestures made by the individuals within the room and translate thegestures to within the virtual 3D space of the 3D cylindrical structure.The sensors may be configured to map a position of a user object usingx, y and z axis coordinates so as to identify a common location betweentwo users. The sensors may then be configured to map a position of eachuser in the space using location information generated from user devices(e.g., mobile device or augmented reality device), so as to generate aviewing region and corresponding perspective view for each user in thethree-dimensional space. The computer may the differentiate between twoor more active regions within the space to display the appropriate imageto each participant.

In some implementations, clinician devices 1610 may include wearabledevices to sense gestures and movements of clinicians. The system may beconfigured to receive motion data (e.g., gyroscope or accelerometerdata) from a wearable or other mobile device associated with a clinician(e.g., the clinician's smart phone or tablet, or virtual realityglasses), and to determine from the motion data the intended gesturesand/or movements of the clinicians. In this manner, each clinician (or adesignated clinician) may use physical gestures to manipulate thecylinders (or other related structures) within the 3D space as if thosecylinders (or structures) were physical structures, and themanipulations may be visualized by all clinicians viewing the 3D space(e.g., using augmented reality glasses or goggles or a computingdevice). Similarly, manipulations of structures within the 3D space byway of a computer interface (e.g., on a mobile device) may also bevisualized by the clinicians using augmented reality.

Each clinician may view (e.g., on a computing device or throughaugmented reality glasses) the same portion of the virtual interface100, or a portion of the interface that may be viewable from the user'sperspective had the three dimensional interface actually existed in thereal world. In a manner similar to the use of a Lazy Susan to rotatefood around a table, a user can rotate data on the cylindrical structurearound the displayed object representation. Alternative manipulations ofelements or data can include pushing or passing elements across thecylinder. In the absence of tablets or glasses, it should also beappreciated that holographic projection of the visual cylindrical healthdata element or elements and other means of two and three-dimensionalpresentation forms are possible.

According to some implementations, one or more cylinders of data mayappear over top of the patient (e.g., in an operating room). Cliniciansof different specialties within the room may view and manipulate thecylinders to share data as a group. For example, one clinician couldrotate a cylinder of data to pass data to another clinician as if usinga Lazy Susan. The other clinician may flag certain data and then passthe flagged data back to the first clinician by rotating the cylinder inthe opposite direction, or by continuing the rotation to a full 360degrees. A clinician may also expand data on a cylinder or raise thecylinder into the air for all clinicians viewing the 3D space to see.

In some implementations, system 1600 and virtual interface 100 make useof either device camera input or visual cues from, for example,augmented reality glasses to self-select data sources for display on thecylindrical health data element. For example, system 1600 may collectreal time motion data from the various clinician devices 1610, 1616 andsensors 1618. If, based on this data, system 1600 detects that a userpoints a camera on a tablet at a ventilator or other medical device, orlooks at an infusion pump/device with visual tools, such as augmentedreality glasses 1616, the system may display relevant data from saiddevice in virtual interface 100 (e.g., on the visual cylindrical healthdata element for the individual patient or the population-centric view).In some implementations, system 1600 may automatically detect suchmachine or device based on image recognition or its known geolocation.Furthermore, augmented reality interactions can enable enhanced viewingof radiographs or other imaging-based health data by enabling largerviewing access. For example, a user viewing an x-ray on the healthcylinder can move a clinician device 1610 such as a tablet around inspace to see a larger view of the x-ray or alternatively utilize ARglasses to view the x-ray on the cylinder in an enlarged manner. Inanother aspect of this disclosure, cross-functional discussion ofpatient status is enhanced by enabling a presenter of the healthcareteam to discuss the patient and simultaneously navigate the displays oraugmented reality as seen by other members of the healthcare teamthrough a review of the patient status. This functionality can betransferred to other members of the healthcare team in turn tofacilitate cross-functional review and coordination of care for thepatient. These transitions and control functions may involvemanipulation of the cylindrical health data element, time-aligned eventsand other visual functions in this disclosure.

FIG. 17 is a conceptual diagram illustrating an example electronicsystem 1700 for generating a virtual reality interface for the displayof patient health data, according to aspects of the subject technology.Electronic system 1700 may be a computing device for execution ofsoftware associated with one or more portions or steps of process 1500,or components and processes provided by FIGS. 1-16 . Electronic system1700 may be representative, in combination with the disclosure regardingFIGS. 1-16 , of the centralized server 1602 or the clinician device(s)1610, 1616 described above. In this regard, electronic system 1700 orcomputing device may be a personal computer or a mobile device such as asmartphone, tablet computer, laptop, PDA, an augmented reality device, awearable such as a watch or band or glasses, or combination thereof, orother touch screen or television with one or more processors embeddedtherein or coupled thereto, or any other sort of computer-relatedelectronic device having network connectivity.

Electronic system 1700 may include various types of computer readablemedia and interfaces for various other types of computer readable media.In the depicted example, electronic system 1700 includes a bus 1708,processing unit(s) 1712, a system memory 1704, a read-only memory (ROM)1710, a permanent storage device 1702, an input device interface 1714,an output device interface 1706, and one or more network interfaces1716. In some implementations, electronic system 1700 may include or beintegrated with other computing devices or circuitry for operation ofthe various components and processes previously described.

Bus 1708 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices ofelectronic system 1700. For instance, bus 1708 communicatively connectsprocessing unit(s) 1712 with ROM 1710, system memory 1704, and permanentstorage device 1702.

From these various memory units, processing unit(s) 1712 retrievesinstructions to execute and data to process in order to execute theprocesses of the subject disclosure. The processing unit(s) can be asingle processor or a multi-core processor in different implementations.

ROM 1710 stores static data and instructions that are needed byprocessing unit(s) 1712 and other modules of the electronic system.Permanent storage device 1702, on the other hand, is a read-and-writememory device. This device is a non-volatile memory unit that storesinstructions and data even when electronic system 1700 is off. Someimplementations of the subject disclosure use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) aspermanent storage device 1702.

Other implementations use a removable storage device (such as a floppydisk, flash drive, and its corresponding disk drive) as permanentstorage device 1702. Like permanent storage device 1702, system memory1704 is a read-and-write memory device. However, unlike storage device1702, system memory 1704 is a volatile read-and-write memory, such arandom access memory. System memory 1704 stores some of the instructionsand data that the processor needs at runtime. In some implementations,the processes of the subject disclosure are stored in system memory1704, permanent storage device 1702, and/or ROM 1710. From these variousmemory units, processing unit(s) 1712 retrieves instructions to executeand data to process in order to execute the processes of someimplementations.

Bus 1708 also connects to input and output device interfaces 1714 and1706. Input device interface 1714 enables the user to communicateinformation and select commands to the electronic system. Input devicesused with input device interface 1714 include, e.g., alphanumerickeyboards and pointing devices (also called “cursor control devices”).Output device interfaces 1706 enables, e.g., the display of imagesgenerated by the electronic system 1700. Output devices used with outputdevice interface 1706 include, e.g., printers and display devices, suchas cathode ray tubes (CRT) or liquid crystal displays (LCD). Someimplementations include devices such as a touchscreen that functions asboth input and output devices.

Also, as shown in FIG. 17 , bus 1708 also couples electronic system 1700to a network (not shown) through network interfaces 1716. Networkinterfaces 1716 may include, e.g., a wireless access point (e.g.,Bluetooth or WiFi) or radio circuitry for connecting to a wirelessaccess point. Network interfaces 1716 may also include hardware (e.g.,Ethernet hardware) for connecting the computer to a part of a network ofcomputers such as a local area network (“LAN”), a wide area network(“WAN”), wireless LAN, or an Intranet, or a network of networks, such asthe Internet. Any or all components of electronic system 1700 can beused in conjunction with the subject disclosure.

These functions described above can be implemented in computer software,firmware or hardware. The techniques can be implemented using one ormore computer program products. Programmable processors and computerscan be included in or packaged as mobile devices. The processes andlogic flows can be performed by one or more programmable processors andby one or more programmable logic circuitry. General and special purposecomputing devices and storage devices can be interconnected throughcommunication networks.

Some implementations include electronic components, such asmicroprocessors, storage and memory that store computer programinstructions in a machine-readable or computer-readable medium(alternatively referred to as computer-readable storage media,machine-readable media, or machine-readable storage media). Someexamples of such computer-readable media include RAM, ROM, read-onlycompact discs (CD-ROM), recordable compact discs (CD-R), rewritablecompact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM,dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g.,DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SDcards, micro-SD cards, etc.), magnetic and/or solid state hard drives,read-only and recordable Blu-Ray® discs, ultra density optical discs,any other optical or magnetic media, and floppy disks. Thecomputer-readable media can store a computer program that is executableby at least one processing unit and includes sets of instructions forperforming various operations. Examples of computer programs or computercode include machine code, such as is produced by a compiler, and filesincluding higher-level code that are executed by a computer, anelectronic component, or a microprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some implementations areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some implementations, such integrated circuits executeinstructions that are stored on the circuit itself.

As used in this specification and any claims of this application, theterms “computer,” “server,” “processor,” and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium” and “computer readable media” are entirelyrestricted to tangible, physical objects that store information in aform that is readable by a computer. These terms exclude any wirelesssignals, wired download signals, and any other ephemeral signals.

To provide for interaction with a user, implementations of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; e.g., feedbackprovided to the user can be any form of sensory feedback, e.g., visualfeedback, auditory feedback, or tactile feedback; and input from theuser can be received in any form, including acoustic, speech, or tactileinput. In addition, a computer can interact with a user by sendingdocuments to and receiving documents from a device that is used by theuser; e.g., by sending web pages to a web browser on a user's clientdevice in response to requests received from the web browser.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back end, middleware, or front endcomponents. The components of the system can be interconnected by anyform or medium of digital data communication, e.g., a communicationnetwork. Examples of communication networks include a local area network(“LAN”) and a wide area network (“WAN”), an inter-network (e.g., theInternet), and peer-to-peer networks (e.g., ad hoc peer-to-peernetworks).

The computing system can include clients and servers. A client andserver are generally remote from each other and may interact through acommunication network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other. In someimplementations, a server transmits data (e.g., an HTML page) to aclient device (e.g., for purposes of displaying data to and receivinguser input from a user interacting with the client device). Datagenerated at the client device (e.g., a result of the user interaction)can be received from the client device at the server.

Those of skill in the art would appreciate that the various illustrativeblocks, modules, elements, components, methods, and algorithms describedherein may be implemented as electronic hardware, computer software, orcombinations of both. To illustrate this interchangeability of hardwareand software, various illustrative blocks, modules, elements,components, methods, and algorithms have been described above generallyin terms of their functionality. Whether such functionality isimplemented as hardware or software depends upon the particularapplication and design constraints imposed on the overall system.Skilled artisans may implement the described functionality in varyingways for each particular application. Various components and blocks maybe arranged differently (e.g., arranged in a different order, orpartitioned in a different way) all without departing from the scope ofthe subject technology.

It is understood that the specific order or hierarchy of steps in theprocesses disclosed is an illustration of example approaches. Based upondesign preferences, it is understood that the specific order orhierarchy of steps in the processes may be rearranged. Some of the stepsmay be performed simultaneously. The accompanying method claims presentelements of the various steps in a sample order, and are not meant to belimited to the specific order or hierarchy presented.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. The previousdescription provides various examples of the subject technology, and thesubject technology is not limited to these examples. Variousmodifications to these aspects will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother aspects. Thus, the claims are not intended to be limited to theaspects shown herein, but is to be accorded the full scope consistentwith the language claims, wherein reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. Pronouns in themasculine (e.g., his) include the feminine and neuter gender (e.g., herand its) and vice versa. Headings and subheadings, if any, are used forconvenience only and do not limit this disclosure.

The term website, as used herein, may include any aspect of a website,including one or more web pages, one or more servers used to host orstore web related content, etc. Accordingly, the term website may beused interchangeably with the terms web page and server. The predicatewords “configured to,” “operable to,” and “programmed to” do not implyany particular tangible or intangible modification of a subject, but,rather, are intended to be used interchangeably. For example, aprocessor configured to monitor and control an operation or a componentmay also mean the processor being programmed to monitor and control theoperation or the processor being operable to monitor and control theoperation. Likewise, a processor configured to execute code can beconstrued as a processor programmed to execute code or operable toexecute code.

The term automatic, as used herein, may include performance by acomputer or machine without user intervention; for example, byinstructions responsive to a predicate action by the computer or machineor other initiation mechanism. The word “example” is used herein to mean“serving as an example or illustration.” Any aspect or design describedherein as “example” is not necessarily to be construed as preferred oradvantageous over other aspects or designs.

A phrase such as an “aspect” does not imply that such aspect isessential to the subject technology or that such aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations.An aspect may provide one or more examples. A phrase such as an aspectmay refer to one or more aspects and vice versa. A phrase such as an“implementation” does not imply that such implementation is essential tothe subject technology or that such implementation applies to allconfigurations of the subject technology. A disclosure relating to animplementation may apply to all implementations, or one or moreimplementations. An implementation may provide one or more examples. Aphrase such as an “implementation” may refer to one or moreimplementations and vice versa. A phrase such as a “configuration” doesnot imply that such configuration is essential to the subject technologyor that such configuration applies to all configurations of the subjecttechnology. A disclosure relating to a configuration may apply to allconfigurations, or one or more configurations. A configuration mayprovide one or more examples. A phrase such as a “configuration” mayrefer to one or more configurations and vice versa.

All structural and functional equivalents to the elements of the variousaspects described throughout this disclosure that are known or latercome to be known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe claims. Moreover, nothing disclosed herein is intended to bededicated to the public regardless of whether such disclosure isexplicitly recited in the claims. No claim element is to be construedunder the provisions of 35 U.S.C. § 112, sixth paragraph, unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” Furthermore, to the extent that the term “include,” “have,” or thelike is used in the description or the claims, such term is intended tobe inclusive in a manner similar to the term “comprise” as “comprise” isinterpreted when employed as a transitional word in a claim.

What is claimed is:
 1. A method for generating a virtual realityinterface to display patient health data, comprising: generating athree-dimensional space including an object representation of at least aportion of a human body and a first series of segmented bands eachhaving a three-dimensional cylindrical curvilinear surface floatingwithin the three-dimensional space, and circumscribing the objectrepresentation and being horizontally separated from each other by anopen space, and each being configured for a continuous presentation ofdata versus time; plotting a two-dimensional data representation of adifferent type of first physiological data versus time along acurvilinear surface of each of the first series of segmented bands;receiving a user selection of a respective segmented band of the firstseries of segmented bands; generating, in response to receiving the userselection, one or more additional segmented bands floating within thethree-dimensional space and partially circumscribing the objectrepresentation above or below, and vertically and longitudinally alignedwith the selected respective segmented band; and displaying, on acurvilinear surface of a respective segmented band of the one or moreadditional segmented bands, a two-dimensional data representation ofsecond physiological data associated with the data being plotted on thecurvilinear surface of the selected respective segmented band.
 2. Themethod of claim 1, wherein at least one two-dimensional datarepresentation of the first physiological data comprises a markerassociated with a portion of the first physiological data, and the userselection corresponds to a selection of the marker.
 3. The method ofclaim 2, further comprising: receiving a data stream corresponding to aphysiological parameter of a patient, wherein the two-dimensional datarepresentation of the second physiological data comprises avisualization of at least a portion of the data stream.
 4. The method ofclaim 1, wherein the one or more additional segmented bands comprises animage viewing dialog configured to display images associated with apatient, and the two-dimensional data representation of secondphysiological data comprises a first image associated with the patient,the method comprising: generating, in response to receiving the userselection, the image viewing dialog floating within thethree-dimensional space, the image viewing dialog comprising an imageviewing area displaying the first image, and circumscribed by multipleicons representative of selectors for other images to display in theimage viewing area; receiving an icon selection of one of the multipleicons; and displaying, response to the icon selection, a second image inthe image viewing area.
 5. The method of claim 1, wherein thethree-dimensional space is generated as part of an augmented reality,and the object representation and the first series of segmented bandsare floating in the augmented reality in relation to at least onephysical object outside of the augmented reality.
 6. The method of claim5, further comprising: receiving motion data from one or more motionsensors; detecting, based on the motion data received from the one ormore motion sensors, a physical gesture performed by a user in aphysical space associated with the augmented reality, wherein the userselection is received responsive to the physical gesture matching apredetermined gesture for selecting the portion of the first series ofsegmented bands.
 7. The method of claim 1, further comprising:determining an occurrence of an event based at least in part on at leastone of the first physiological data; generating a differentrepresentation of the occurrence of the event for including in arespective two-dimensional data representation of the firstphysiological data, wherein the different representation of theoccurrence of the event comprises a two-dimensional shape that isoverlaid along a portion of the respective two-dimensional datarepresentation of the first physiological data coinciding with theoccurrence of the event; receiving a second user selection of theportion of the respective two-dimensional data representation of thefirst physiological data coinciding with the occurrence of the event;and in response to the second user selection, generating a secondvisualization of a second portion of a data stream corresponding to theportion of the respective two-dimensional data representation of thefirst physiological data coinciding with the occurrence of the event. 8.The method of claim 1, wherein a first segmented band of the firstseries of segmented bands and a second segmented band of the one or moreadditional segmented bands are configured to become vertically alignedin time and time-locked such that if the first and second segmentedbands are viewed together and one is rotated to scroll through time, theother will rotate along with it so that events occurring on each surfaceare viewed together at the same time.
 9. A system comprising: aprocessor; a memory device containing instructions which, when executedby the processor cause the processor to: generate a three-dimensionalspace including an object representation of at least a portion of ahuman body and a first series of segmented bands each having athree-dimensional cylindrical curvilinear surface floating within thethree-dimensional space and circumscribed about at least a portion ofthe object representation, and circumscribing the object representationand being horizontally separated from each other by an open space, andeach being configured for a continuous presentation of data versus time;plot a two-dimensional data representation of a different type of firstphysiological data, versus time along a curvilinear surface of each ofthe first series of segmented bands; receive a user selection of arespective segmented band of the first series of segmented bands;generate, in response to receiving the user selection, one or moreadditional segmented bands floating within the three-dimensional spaceand partially circumscribing the object representation above or below,and vertically and longitudinally aligned with the selected respectivesegmented band; and display, on a curvilinear surface of a respectivesegmented band of the one or more additional segmented bands, atwo-dimensional data representation of second physiological dataassociated with the data being plotted on the curvilinear surface of theselected respective segmented band.
 10. The system of claim 9, whereinat least one two-dimensional data representation of the firstphysiological data comprises a marker associated with a portion of thefirst physiological data, and the user selection corresponds to aselection of the marker.
 11. The system of claim 10, wherein each of thefirst series of segmented bands has a height dimension and a lengthdimension circumscribed about the at least a portion of the objectrepresentation, the height dimension and length dimension togetherdefining a two-dimensional data area for the presentation of data to auser viewing a segmented band of the first series of segmented bands inthe three-dimensional space.
 12. The system of claim 9, wherein at leastone of the one or more additional segmented bands comprises an imageviewing dialog configured to display images associated with a patient,and the two-dimensional data representation of second physiological datacomprises a first image associated with the patient, wherein theinstructions, when executed, further cause the processor to: generate,in response to receiving the user selection, the image viewing dialogfloating within the three-dimensional space, the image viewing dialogcomprising an image viewing area displaying the first image, andcircumscribed by multiple icons representative of selectors for otherimages to display in the image viewing area; receive an icon selectionof one of the multiple icons; and display, responsive to the iconselection, a second image in the image viewing area.
 13. The system ofclaim 9, wherein the three-dimensional space is generated as part of anaugmented reality, and the object representation and the first series ofsegmented bands are floating in the augmented reality in relation to atleast one physical object outside of the augmented reality.
 14. Thesystem of claim 13, wherein the instructions, when executed, furthercause the processor to: receive motion data from one or more motionsensors; detect, based on the motion data received from the one or moremotion sensors, a physical gesture performed by a user in a physicalspace associated with the augmented reality, wherein the user selectionis received responsive to the physical gesture matching a predeterminedgesture for selecting the portion of the first series of segmentedbands.
 15. The system of claim 9, wherein a first segmented band of thefirst series of segmented bands and a second segmented band of the oneor more additional segmented bands are configured to become verticallyaligned in time and time-locked such that if the first and secondsegmented bands are viewed together and one is rotated to scroll throughtime, the other will rotate along with it so that events occurring oneach surface are viewed together at the same time.
 16. A non-transitorycomputer-readable medium comprising instructions, which when executed bya computing device, cause the computing device to perform operationscomprising: generating a three-dimensional space including an objectrepresentation of at least a portion of a human body and a first seriesof segmented bands each having a three-dimensional cylindricalcurvilinear surface floating within the three-dimensional space, andcircumscribing the object representation and being horizontallyseparated from each other by an open space, and each being configuredfor a continuous presentation of data versus time; plotting atwo-dimensional data representation of a different type of firstphysiological data versus time along a curvilinear surface of each ofthe first series of segmented bands; receiving a user selection of arespective segmented band of the first series of segmented bands;generating, in response to receiving the user selection, one or moreadditional segmented bands floating within the three-dimensional spaceand partially circumscribing the object representation above or below,and vertically and longitudinally aligned with the selected respectivesegmented band; and displaying, on a curvilinear surface of a respectivesegmented band of the one or more additional segmented bands, atwo-dimensional data representation of second physiological dataassociated with the data being plotted on the curvilinear surface of theselected respective segmented band.